Simple randomized algorithms for online learning with kernels
نویسندگان
چکیده
In online learning with kernels, it is vital to control the size (budget) of the support set because of the curse of kernelization. In this paper, we propose two simple and effective stochastic strategies for controlling the budget. Both algorithms have an expected regret that is sublinear in the horizon. Experimental results on a number of benchmark data sets demonstrate encouraging performance in terms of both efficacy and efficiency.
منابع مشابه
Online Learning with Multiple Operator-valued Kernels
We consider the problem of learning a vector-valued function f in an online learning setting. The function f is assumed to lie in a reproducing Hilbert space of operator-valued kernels. We describe two online algorithms for learning f while taking into account the output structure. A first contribution is an algorithm, ONORMA, that extends the standard kernel-based online learning algorithm NOR...
متن کاملApproximate Stochastic Subgradient Estimation Training for Support Vector Machines
Subgradient algorithms for training support vector machines have been quite successful for solving largescale and online learning problems. However, they have been restricted to linear kernels and strongly convex formulations. This paper describes efficient subgradient approaches without such limitations. Our approaches make use of randomized low-dimensional approximations to nonlinear kernels,...
متن کاملOnline Learning for Structural Kernels
While online learning techniques have existed since Rosenblatt’s introduction of the Perceptron in 1957, there has been a renewed interest lately due to the need for efficient classification algorithms. Additionally, kernel techniques have allowed online learning to be extended to problems whose classes are not linearly separable in their native space. Online algorithms typically result in poor...
متن کاملLearning Additive Models Online with Fast Evaluating Kernels
We develop three new techniques to build on the recent advances in online learning with kernels. First, we show that an exponential speed-up in prediction time per trial is possible for such algorithms as the Kernel-Adatron, the Kernel-Perceptron, and ROMMA for specific additive models. Second, we show that the techniques of the recent algorithms developed for online linear prediction when the ...
متن کاملBudget Online Multiple Kernel Learning
Online learning with multiple kernels has gained increasing interests in recent years and found many applications. For classification tasks, Online Multiple Kernel Classification (OMKC), which learns a kernel based classifier by seeking the optimal linear combination of a pool of single kernel classifiers in an online fashion, achieves superior accuracy and enjoys great flexibility compared wit...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Neural networks : the official journal of the International Neural Network Society
دوره 60 شماره
صفحات -
تاریخ انتشار 2014